{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Python机器学习实训课程之线性回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 任务描述\n",
    "\n",
    "本关任务：掌握线性回归的算法原理，用线性回归算法对测试集进行预测。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 相关知识\n",
    "\n",
    "#### 1.线性回归原理\n",
    "##### 简单线性回归\n",
    "\n",
    "在生活中，我们常常能碰到这么一种情况，一个变量会跟着另一个变量的变化而变化，如圆的周长与半径的关系，当圆的半径确定了，那么周长也就确定了。还有一种情况就是，两个变量之间看似存在某种关系，但又没那么确定，如青少年的身高与体重，他们存在一种近似的**线性**关系：\n",
    "\n",
    "*身高/cm = 体重/kg +105*\n",
    "\n",
    "但是，并不是每个青少年都符合这个公式，只能说每个青少年的身高体重都存在这么一种近似的线性关系。那么，到底什么是线性回归呢？假如我们将青少年的身高和体重值作为坐标，不同人的身高体重就会在平面上构成不同的坐标点，然后用一条直线，尽可能的去拟合这些点，这就是简单的**线性回归**。\n",
    "\n",
    "![](https://www.educoder.net/api/attachments/313533)\n",
    "\n",
    "\n",
    "\n",
    "简单的线性回归模型如下：\n",
    "\n",
    "$$y=wx+b$$\n",
    "\n",
    "其中 $x$ 表示特征值(如：体重值)，$w$ 表示权重，$b$ 表示偏置，$y$ 表示标签(如：身高值)。\n",
    "\n",
    "##### 多元线性回归\n",
    "\n",
    "在简单线性回归中，一个变量会随另一个变量的变化而变化。但是生活中，还有很多变量，可能会随多个变量的变化而变化，比如房价，影响它的因素可能有：房屋面积、地理位置等等。如果我们要给它们建立出近似的线性关系，就需要使用多元线性回归，多元线性回归模型如下：\n",
    "\n",
    "$$y=b+w_1x_1+w_2x_2+...+w_nx_n$$\n",
    "\n",
    "其中 $x_i$ 表示第 $i$ 个特征值，$w_i$ 表示第 $i$ 个特征对应的权重，$b$ 表示偏置，$y$ 表示标签。\n",
    "\n",
    "多元线性回归模型还有另一种等价的表达形式：\n",
    "\n",
    "$$y=\\theta_0x_0+\\theta_1x_1+\\theta_2x_2+...+\\theta_nx_n$$\n",
    "\n",
    "令 $x_0=1$，则 $\\theta_0$ 相当于偏执 $b$。\n",
    "\n",
    "对线性回归模型，假设训练集中 m 个训练样本，每个训练样本中有 n 个特征，可以采用矩阵的表示方法，则多元线性回归模型还可以表示为：\n",
    "\n",
    "$$Y=\\theta X$$\n",
    "\n",
    "其中$\\theta=(\\theta_0,\\theta_1,\\theta_2,...,\\theta_n)$。\n",
    "\n",
    "其损失函数可以表示为：\n",
    "\n",
    "$$J(\\theta)=(Y-\\hat{Y})^2=(Y-\\theta X)^2=(Y-\\theta X)^T(Y-\\theta X)$$\n",
    "\n",
    "其中，标签 $Y$ 为 (m,1) 的矩阵，训练特征 $X$ 为 (m,n+1) 的矩阵(**列数为n+1，是因为需要添加一列 $x_0$， 并且这一列的值都为 1**)，回归系数 $\\theta$ 为 (n+1,1) 的矩阵，对 $\\theta$ 求导，并令其导数等于 0 ，可以得到 $X^T(Y-\\theta X)=0$。所以，最优解为：\n",
    "\n",
    "$$\\theta = (X^TX)^{-1}X^TY$$\n",
    "\n",
    "这个就是正规方程解，我们可以通过最优方程解，直接求得我们所需要的参数 $\\theta$。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在算法中，经常会用到矩阵知识，为了让大家回顾矩阵相乘的知识，对矩阵有更好的了解，下面是一个矩阵应用的案例。\n",
    "#### 2.矩阵应用举例\n",
    "某公司有四个工厂，分布在不同地区，同时生产三种产品，如下表所示：\n",
    "\n",
    "|工厂\\产品|P1|P2|P3|\n",
    "|:---:|:---:|:---:|:---:|\n",
    "|甲| 5 | 2 | 4 |\n",
    "|乙|3|8|2|\n",
    "|丙|6|0|4|\n",
    "|丁|0|1|6|\n",
    "\n",
    "其中，四行分别表示甲乙丙丁四个工厂的生产情况，三列分别表示三种产品 P1、P2、P3 的产量。可以用矩阵表示为 A=$\\begin{pmatrix}5 & 2 &4\\\\ 3 & 8&2\\\\6&0&4\\\\0&1&6 \\end{pmatrix}$。\n",
    "\n",
    "再设矩阵 B=$\\begin{pmatrix}2 & 4\\\\ 1 & 3\\\\3&2 \\end{pmatrix}$，其中第一列表示三种产品的单件利润，第二列表示三种产品的单价体积。\n",
    "\n",
    "C = AB = $\\begin{pmatrix}5 & 2 &4\\\\ 3 & 8&2\\\\6&0&4\\\\0&1&6 \\end{pmatrix}$$\\begin{pmatrix}2 & 4\\\\ 1 & 3\\\\3&2 \\end{pmatrix}$ = $\\begin{pmatrix}24 & 44 \\\\ 20 & 40\\\\24&32\\\\19&15 \\end{pmatrix}$\n",
    "\n",
    "矩阵 C，是由矩阵 A 与矩阵 B 相乘得到的。矩阵 C 的第一列数据分别表示四个工厂的利润，第二列分别表示四个工厂需要的存储空间。\n",
    "\n",
    "以上是简单的矩阵乘法，手算比较麻烦，可以通过 numpy 直接实现矩阵的乘法。在算法的实现代码里，会用到了很多 numpy 的知识，下面介绍一些在算法代码中将要用到的 numpy 基础知识。\n",
    "#### 3.numpy基础知识"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[5 2 4]\n",
      " [3 8 2]\n",
      " [6 0 4]\n",
      " [0 1 6]]\n",
      "--------\n",
      "[[2 4]\n",
      " [1 3]\n",
      " [3 2]]\n"
     ]
    }
   ],
   "source": [
    "# np.mat用于生成矩阵\n",
    "# 生成两个矩阵\n",
    "import numpy as np\n",
    "A= np.mat([[5,2,4],\n",
    "           [3,8,2],\n",
    "           [6,0,4],\n",
    "           [0,1,6]]) # 创建矩阵A\n",
    "B=np.mat([[2,4],\n",
    "           [1,3],\n",
    "           [3,2]]) # 创建矩阵B\n",
    "print(A)\n",
    "print('--------')\n",
    "print(B)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[5, 3, 6, 0],\n",
       "        [2, 8, 0, 1],\n",
       "        [4, 2, 4, 6]])"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# .T可以得到矩阵的转置\n",
    "A.T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看出行和列进行了转换。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[24, 34],\n",
       "        [20, 40],\n",
       "        [24, 32],\n",
       "        [19, 15]])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# *可以实现矩阵乘法\n",
    "A*B"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1 2 3]\n",
      " [2 2 3]\n",
      " [2 3 4]]\n",
      "--------\n",
      "[[-1.  1.  0.]\n",
      " [-2. -2.  3.]\n",
      " [ 2.  1. -2.]]\n"
     ]
    }
   ],
   "source": [
    "# .I用于求矩阵的逆矩阵\n",
    "m = np.mat([[1,2,3], [2,2,3], [2,3,4]])   #定义一个方阵\n",
    "print(m)\n",
    "print('--------')\n",
    "print(m.I) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[1., 0., 0.],\n",
       "        [0., 1., 0.],\n",
       "        [0., 0., 1.]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#由矩阵知识，可知一个矩阵和它的逆矩阵相乘，会得到一个单位矩阵\n",
    "m*m.I"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[ 0  1]\n",
      "  [ 2  3]]\n",
      "\n",
      " [[ 4  5]\n",
      "  [ 6  7]]\n",
      "\n",
      " [[ 8  9]\n",
      "  [10 11]]\n",
      "\n",
      " [[12 13]\n",
      "  [14 15]]\n",
      "\n",
      " [[16 17]\n",
      "  [18 19]]]\n",
      "--------\n",
      "[-2. -2. -2. -2. -2.]\n"
     ]
    }
   ],
   "source": [
    "#numpy.linalg.det() 可以计算数组的行列式\n",
    "a=np.reshape(np.arange(20),(5,2,2))\n",
    "b=np.linalg.det(a)\n",
    "print(a)\n",
    "print('--------')\n",
    "print(b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.线性回归的实现代码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 定义类LinearRegression\n",
    "class LinearRegression :\n",
    "    # 初始化线性回归模型\n",
    "    def __init__(self):\n",
    "        self.theta = None\n",
    "        \n",
    "    # 通过代码实现theta的求解    \n",
    "    def fit(self,xArr,yArr):\n",
    "        '''\n",
    "        input:xArr(ndarray): 特征值\n",
    "              yArr(ndarray): 标签值\n",
    "        output:无\n",
    "        '''\n",
    "        xMat = np.mat(xArr) # 将数据转化为矩阵\n",
    "        yMat = np.mat(yArr).T # 将数据转化为矩阵，并进行转置\n",
    "        xTx = xMat.T*xMat # 矩阵xMat转置后相乘\n",
    "        # 判断矩阵xTx是否是奇异矩阵\n",
    "        if np.linalg.det(xTx) == 0.0:\n",
    "            print(\"这个矩阵是奇异的，不可求逆\")\n",
    "            return\n",
    "        self.theta = xTx.I * (xMat.T*yMat) # 根据最优解的求解公式，求theta值\n",
    "    \n",
    "    # 对测试数据进行预测\n",
    "    def predict(self,test_data):\n",
    "        '''\n",
    "        input:test_data(ndarray):测试样本\n",
    "        output:predict(matrix):预测结果\n",
    "        '''        \n",
    "        test_data = np.mat(test_data) # 将数据转化为矩阵\n",
    "        predict = test_data*self.theta #通过得到的theta值，对数据进行预测\n",
    "        return predict"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 5.案例应用\n",
    "##### 读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'pandas'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-8-b79c1bc542a4>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mpandas\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      2\u001b[0m \u001b[0mdata\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_table\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'/data/shixunfiles/b8a973b880e78ba536e67c243fc1932c_1577257252312.txt'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mheader\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mhead\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'pandas'"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "data = pd.read_table('/data/shixunfiles/b8a973b880e78ba536e67c243fc1932c_1577257252312.txt',header=None)\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从数据可以看出，第一列的值总是等于1，对应的就是线性回归里的x0；第二列的值相当于线性回归里的x1；第三列的值相当于线性回归里的y。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 通过图形查看数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#导入画图模块\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "#展示数据\n",
    "plt.scatter(data.iloc[:,1],data.iloc[:,2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过图形可以看出，y值大致随着x的增长呈现线性增长的趋势。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 查看数据大小"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(data.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 提取特征值和真实值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#提取特征值\n",
    "X=data.iloc[:,:-1].values\n",
    "#提取标签值\n",
    "y=data.iloc[:,-1].values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 建立模型，并训练模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#建立模型\n",
    "lr=LinearRegression()\n",
    "#训练模型\n",
    "lr.fit(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#####  对数据进行预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#为了求拟合曲线，因此使用原始数据进行预测\n",
    "ypredict=lr.predict(X)\n",
    "#查看预测的前10条数据\n",
    "print(ypredict[:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 通过图形查看拟合效果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.scatter(data.iloc[:,1],data.iloc[:,2])\n",
    "plt.plot(data.iloc[:,1],ypredict,color='black')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "图中，蓝色的点为真实值，黑色的线为拟合直线。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 自由编程"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
